1. Field of the Invention
The present invention relates generally to interactive computer graphics, and more particularly to managing multiple frame buffers in an interactive computer graphics system.
2. Related Art
A constant frame-rate, interactive computer graphics application is a computer application that endeavors to generate a new image frame every N video frame periods, where N is typically a small constant integer. A video frame period is equal to the time needed to display an image frame a single time on a computer display. A video frame period includes any overhead associated with the particular computer display being used, such as vertical retrace associated with raster displays. A frame time is equal to N video frame periods. A frame time refers to the amount of time that each image frame is displayed on the computer display.
If a constant frame-rate application is unable to generate a frame within one frame time, then the application is said to have dropped a frame. Dropping a frame detracts from the graphical presentation and interactiveness of the application. Accordingly, constant frame-rate applications make every effort to avoid dropping frames.
In particular, constant frame-rate applications monitor the utilization of graphics resources during the rendering of the current frame in order to predict the behavior of future frames. If such prediction indicates that a future frame may be dropped due to an overload condition, then the rendering complexity of that frame is reduced (e.g., by using models of lower geometric resolution) so as to avoid the overload condition.
Many modern interactive graphics systems utilize a double buffered system that includes two frame buffers. Such systems include those that support OpenGL (GL stands for graphics library), which is a software interface to graphics hardware.
In a double buffered system, the frame (called Frame M) stored in one of the buffers is displayed. This buffer is called the Front Buffer. The other buffer is used to construct (i.e., generate or draw or render) the next frame (called Frame M+1). This buffer is called the Back Buffer. At the conclusion of Frame M's frame time, a Swap command is executed to logically convert the Back Buffer to the Front Buffer. This results in Frame M+1 being displayed.
Since double buffered systems include only two frame buffers, Frame M+1 must be rendered during the frame time that Frame M is displayed. Otherwise, a frame (i.e., Frame M+1) will be dropped.
However, it is impossible to precisely predict the time required to generate a frame. Constant frame-rate applications compensate for this imprecision by underutilizing the rendering hardware in order to insure that frames are not dropped. This is not an ideal solution, however, since it is preferable for resolution purposes to utilize the rendering hardware to the greatest extent possible.
A conventional solution to this problem is to utilize more than two frame buffers. The use of three or more frame buffers enables a constant frame-rate application to take longer than one frame time to generate a frame, without causing a frame to be dropped. This solution has been proposed and implemented in the MBX extension of the X Window System. MBX is an extension of OpenGL. MBX stands for multibuffer extension.
There are two significant drawbacks to conventional multibuffer (greater than two buffer) systems. The first deals with the naming of the frame buffers, or in the larger sense, answering the question of how an application will have access to the buffers. According to MBX, each frame buffer is named. That is, MBX allows each frame buffer to be rendered and displayed in any order, and at any time. Thus, MBX cannot be used effectively without explicit, substantial, and skilled application reprogramming.
The second drawback deals with the potential increase in render-to-display latency that results from the use of multiple frame buffers. Render-to-display latency refers to the time from the moment that rendering of a frame is begun to the time that the frame is displayed. Large render-to-display latency values detract from the interactiveness of a graphics system, and are therefore guarded against with as much zeal as dropped frames are avoided. (Note that one consequence of dropping a frame is increased latency.) Double buffered systems inherently limit latency to one frame time. In the absence of artificially imposed constraints, a multibuffer system with greater than two frame buffers could increase render-to-display latency to significantly greater than one frame time.
The increased render-to-display latency associated with a three buffer system (with Buffers A, B, and C) is illustrated in a timing diagram 102 shown in FIGS. 1A and 1B. FIG. 1 illustrates the orientation of FIGS. 1A and 1B. In the example of FIGS. 1A and 1B, a video frame period is equal to a frame time (i.e., N is equal to one). For illustrative purposes, each video frame period or frame time is shown as including eight equal duration time intervals. For example, video frame period P2 runs from time t8 to time t15. It should be understood that such segmentation of video frame periods as shown in FIGS. 1A and 1B is done for illustrative purposes only. In practice, the temporal segmentation of video frame periods is not as neat and clearcut as that shown in FIGS. 1A and 1B.
A Frame Being Displayed row 110 indicates the frame that is being displayed for the corresponding time interval indicated in the Time row 106, and a Buffer Being Displayed row 108 indicates the buffer in which the displayed frame is stored. A Frame Being Rendered row 114 indicates the frame that is being rendered during the corresponding time interval indicated in the Time row 106, and a Buffer Being Rendered row 112 indicates the buffer in which the rendering is taking place.
For purposes of the example shown in FIGS. 1A and 1B, it is assumed that 87.5% of a video frame period (i.e., 7 out of 8 time intervals) is required to display a frame. It is assumed that 12.5% of a video frame period (i.e., 1 out of 8 time intervals) is required for overhead processing required by the particular display unit being used. Thus, frame F0 stored in Buffer A is shown as being displayed during time intervals t0-t6 of video frame period P1. Time interval t7 is used for overhead processing.
It is also assumed that 75% of a video frame period (i.e., 6 out of 8 time intervals) is required to render a frame. Thus, frame F3 is rendered in Buffer A during time intervals t10-t15. Conventional multibuffer systems allow the next frame to be rendered as soon as rendering of the previous frame is complete (assuming that a buffer is available). Thus, during time intervals t0-t21, frames F1, F2, F3, and F4 are rendered immediately one after another. However, frame F5 cannot be rendered in Buffer C until time interval t23, since Buffer C is not available until time interval t23 (note that Buffer C is used for display purposes up to and including time interval t22).
In the three buffer scenario of FIGS. 1A and 1B, render-to-display latency increases over time because: (1) the rendering hardware is not fully utilized (as indicated by the fact that rendering occurs during only 75% of a video frame period); and (2) the rendering of the next frame is allowed to begin immediately after the rendering of the preceding frame. Thus, the latency 116 of frame F2 is equal to 12 time intervals (1.5 video frame periods), whereas the latency 117 of Frame F3 is equal to 14 time intervals (1.75 video frame periods). Latency is ultimately bound by the availability of frame buffers for rendering. In the steady state, the render-to-display latency is equal to 17 time intervals (just over two video frame periods), as illustrated by the latency 118 for frame F6.
Thus, what is required is a multibuffer interactive graphics system having three or more frame buffers, wherein the system does not suffer from the naming problem or the latency problem described above.